A Big Five-Based Multimethod Social and Emotional Skills Assessment: The Mosaic™ by ACT® Social Emotional Learning Assessment.

A focus on implementing social and emotional (SE) learning into curricula continues to gain popularity in K-12 educational contexts at the policy and practitioner levels. As it continues to be elevated in educational discourse, it becomes increasingly clear that it is important to have reliable, validated measures of students' SE skills. Here we argue that framework and design are additional important considerations for the development and selection of SE skill assessments. We report the reliability and validity evidence for The Mosaic™ by ACT® Social Emotional Learning Assessment, an assessment designed to measure SE skills in middle and high school students that makes use of a research-based framework (the Big Five) and a multi-method approach (three item types including Likert, forced choice, and situational judgment tests). Here, we provide the results from data collected from more than 33,000 students who completed the assessment and for whom we have data on various outcome measures. We examined the validity evidence for the individual item types and the aggregate scores based on those three. Our findings support the contribution of multi-method assessment and an aggregate score. We discuss the ways the field can benefit from this or similarly designed assessments and discuss how the assessment results can be used by practitioners to promote programs aimed at stimulating students' personal growth.

Adjectives vs. Statements in Forced Choice and Likert Item Types: Which Is More Resistant to Impression Management in Personality Assessment?

Our objective was to compare individuals' ability to intentionally make a positive impression when responding to a Five-Factor Model personality measure under adjective vs. statement and forced choice vs. Likert conditions. Participants were 1,798 high school students who were randomly assigned to either a condition receiving normal instructions or instructions to make a positive impression. We compared the groups' scores and validity estimates under the various conditions. Although impression management occurred on all item types, participants could more easily manipulate their responses to Likert items vs. forced choice items, and statements vs. adjectives. Item type made little difference in terms of convergent and discriminant validity and criterion-related validity for all outcomes but one, ACT scores, which suggests cognitive ability plays a role in impression management ability.

Measuring achievement striving via a situational judgment test: the value of additional context / Medición del esfuerzo hacia el logro mediante de un test de juicio situacional: el valor del contexto adicional

The study extends personality and situational judgment test (SJT) research by using an SJT to measure achievement striving in a contextualized manner. Employed students responded to the achievement striving SJT, traditional personality scales, and workplace performance measures. The SJT was internally consistent, items loaded on a single factor, and scores converged with other measures of achievement striving. The SJT provided incremental criterion-related validity for the performance criteria beyond less-contextualized achievement striving measures. Findings suggest that achievement-related work scenarios may provide additional criterion-relevant information not captured by measures that are less contextualized

Este estudio extiende la investigación sobre los tests de juicio situacional (TJS) y la personalidad, usando un TJS para medir el esfuerzo hacia el logro de una manera contextualizada. Estudiantes con empleo respondieron el TJS de esfuerzo hacia el logro, escalas típicas de personalidad y medidas de desempeño en el trabajo. El TJS fue internamente consistente, los ítems cargaron en un único factor y las puntuaciones convergieron con otras medidas de esfuerzo hacia el logro. El TJS añadió validez relacionada con el criterio, para los criterios de desempeño, a la validez de las medidas menos contextualizadas de esfuerzo hacia el logro. Los hallazgos sugieren que escenarios de trabajo relacionados con el logro pueden proporcionar información adicional relevante para el criterio no capturada por medidas menos contextualizadas

Scale for the study of nursing students' perception of intimate partner violence: Adaptation and validation.

BACKGROUND: Intimate partner violence (IPV) is one of the most important challenges facing today's society. Health professionals, and nurses in particular, play a leading role in addressing this problem. Having an instrument to study Nursing students' perception of this type of violence may enable appropriate and necessary educational strategies for their pre-service training and help them identify and address IPV in their future careers. OBJECTIVES: To adapt and validate the IPV scale of Beccaria et al. (2011) in order to study nursing students' perception of IPV. METHOD: The original instrument (63 items) underwent back-translation. Statistical and metrical analysis in a sample of 1064 students of the four-year Nursing degree program at four public universities in Catalonia (Spain) were based on their responses to the Spanish version. RESULTS: The results of the exploratory factorial analysis and subsequent confirmatory analysis showed that the data had an adequate fit for a four-factor model. The reliability analysis showed adequate internal consistency for each subscale in the instrument: Education (0.83); Identification of the Victim (0.72); Nursing Role and Values (0.78), and Identification of the Perpetrator (0.66). CONCLUSION: The translation, cultural adaptation and validation process of the original IPV scale resulted in a Spanish-language instrument (IPV scale Spanish version) with 32 items in four subscales. We concluded that the Spanish version of the instrument is reliable and valid, and that its implementation would enable the assessment of nursing students' perception of IPV situations.

Fluid Ability (Gf) and Complex Problem Solving (CPS).

Complex problem solving (CPS) has emerged over the past several decades as an important construct in education and in the workforce. We examine the relationship between CPS and general fluid ability (Gf) both conceptually and empirically. A review of definitions of the two factors, prototypical tasks, and the information processing analyses of performance on those tasks suggest considerable conceptual overlap. We review three definitions of CPS: a general definition emerging from the human problem solving literature; a more specialized definition from the "German School" emphasizing performance in many-variable microworlds, with high domain-knowledge requirements; and a third definition based on performance in Minimal Complex Systems (MCS), with fewer variables and reduced knowledge requirements. We find a correlation of 0.86 between expert ratings of the importance of CPS and Gf across 691 occupations in the O*NET database. We find evidence that employers value both Gf and CPS skills, but CPS skills more highly, even after controlling for the importance of domain knowledge. We suggest that this may be due to CPS requiring not just cognitive ability but additionally skill in applying that ability in domains. We suggest that a fruitful future direction is to explore the importance of domain knowledge in CPS.

Controlling social desirability may attenuate faking effects: a study with aggression measures / Controlar la deseabilidad social puede atenuar los efectos del falseamiento: un estudio con medidas de agresividad

Background: Several studies have been conducted to better understand what happens with personality scores when faking occurs, but very few use socially undesirable trait measures such as aggression. The aim of the present research is twofold: (a) we aim to apply a General Factorial-Analytic procedure to aggression scales and determine whether it can correct for faking effects; (b) we aim to test the impact that individual differences can have on change scores due to faking. Method: Participants were 371 undergraduate students. Of these, 215 answered the questionnaires twice, under neutral conditions and under faking-motivating conditions. 156 were the control group who answered the questionnaires twice, both times under neutral conditions. Results: The mean comparison tests as well as the repeated measures ANOVA showed significant results. Individual differences played an important role in all the scales except in physical aggression. Conclusions: The results showed that the procedure does correct for faking effects and that individual differences have an important impact on the change scores due to faking, except in the most undesirable Physical aggression measure, which was hardly affected (AU)

Antecedentes: varios estudios han intentado entender qué sucede con las puntuaciones de las medidas de personalidad cuando se produce falseamiento, pero pocos han utilizado medidas socialmente indeseables como la agresividad. El presente estudio tiene dos objetivos principales: (a) se quiere aplicar el método Analítico Factorial General a las escalas de agresividad para determinar si el método puede corregir los efectos del falseamiento; (b) se quiere comprobar el impacto de las diferencias individuales en las puntuaciones de cambio debido a falseamiento. Método: 371 estudiantes universitarios participaron en el estudio. De ellos, 215 respondieron el cuestionario dos veces, bajo condiciones neutras y bajo condición de falseamiento. 156 participantes formaron el grupo control, contestando dos veces el cuestionario bajo condiciones neutras. Resultados: tanto las pruebas de comparación de medias como el ANOVA de medidas repetidas obtuvieron resultados significativos. Las diferencias individuales tuvieron un papel importante en todas las escalas excepto en la de agresividad física. Conclusiones: los resultados muestran que el método corrige los efectos del falseamiento y que las diferencias individuales tienen un papel importante en las puntuaciones de cambio debido a falseamiento, con la excepción de la medida de agresividad más indeseable, la agresividad física, que casi no se ve afectada (AU)

Controlling social desirability may attenuate faking effects: a study with aggression measures.

BACKGROUND: Several studies have been conducted to better understand what happens with personality scores when faking occurs, but very few use socially undesirable trait measures such as aggression. The aim of the present research is twofold: (a) we aim to apply a General Factorial-Analytic procedure to aggression scales and determine whether it can correct for faking effects; (b) we aim to test the impact that individual differences can have on change scores due to faking. METHOD: Participants were 371 undergraduate students. Of these, 215 answered the questionnaires twice, under neutral conditions and under faking-motivating conditions. 156 were the control group who answered the questionnaires twice, both times under neutral conditions. RESULTS: The mean comparison tests as well as the repeated measures ANOVA showed significant results. Individual differences played an important role in all the scales except in physical aggression. CONCLUSIONS: The results showed that the procedure does correct for faking effects and that individual differences have an important impact on the change scores due to faking, except in the most undesirable Physical aggression measure, which was hardly affected.

Measuring anxiety in visually-impaired people: a comparison between the linear and the nonlinear IRT approaches

La presente investigación tiene dos intereses principales. En primer lugar, se evalúan algunas cuestiones pendientes acerca de las propiedades psicométricas del CTAC (cuestionario de ansiedad para ciegos y deficientes visuales) mediante la teoría de respuesta a los items (TRI). En segundo lugar, en las tres propiedades evaluadas: precisión de medida, sensibilidad al cambio e índices de ajuste de la persona, el modelo lineal se compara con el modelo de respuesta graduada (GRM) y los resultados obtenidos sirven además como ilustración de las posibilidades y ventajas de los modelos TRI. Los participantes son 670 ciegos o personas con deficiencia visual de diferentes ciudades españolas. Los resultados mostraron que las puntuaciones del CTAC son suficientemente precisas para el uso al que están destinadas, y que los participantes son generalmente consistentes en sus respuestas. Los datos eran apropiados para los dos modelos puestos a prueba llevando a resultados similares en lo referente a las estimaciones en los niveles en el rasgo, con la excepción de los participantes extremos que fueron mejor evaluados por el modelo lineal. El GRM mostró mayores ventajas en la evaluación de la precisión de la medida y ambos modelos mostraron alta sensibilidad al cambio alrededor del punto de corte. Los resultados concernientes al ajuste de la persona también fueron similares en ambos modelos(AU)

The present study has two main interests. First, some pending issues about the psychometric properties of the CTAC (an anxiety questionnaire for blind and visually-impaired people) are assessed using item response theory (IRT). Second, the linear model is compared to the graded response model (GRM) in terms of measurement precision, sensitivity to change, and person fit, and the results are also used to illustrate the functioning and advantages of IRT models. The participants were 670 blind or visually-impaired people from different Spanish cities. The results showed that the CTAC scores are accurate enough for practical purposes, and that respondents are quite consistent in their responses. Model-data fit was acceptable in both cases, and both models lead to similar results regarding the trait estimates, with the exception of extreme respondents who were better assessed with the linear model. The GRM assessed measurement precision better, and both models showed high sensitivity to change around cut-off values. Person-fit results were also similar in both models(AU)

The impact of social desirability on psychometric measures of aggression / El efecto de la deseabilidad en las medidas psicométricas de agresividad

Although many studies have focused on the effects of social desirability in personality measures, few have analysed its effects on such highly undesirable behaviour as aggressiveness. The present study analyzes the impact of social desirability on measures of direct and indirect aggression and on the relationships between both kinds of aggression with impulsivity, using a method that enables the content factors of the measures to be isolated from social desirability. Results showed that aggression measures are highly affected by social desirability and that the relationships between the two forms of aggression and impulsivity are due to the content measured by the tests and not to a common social desirability factor (AU)

Aunque numerosos estudios han analizado los efectos de la deseabilidad social en las medidas de personalidad, muy pocos se han centrado en los efectos de la misma en un comportamiento fuertemente reprobado por la sociedad como es la agresividad. El estudio que presentamos analiza el impacto de la deseabilidad social en las medidas de agresividad directa e indirecta, así como en sus relaciones con la impulsividad utilizando un método que permite obtener puntuaciones en los factores de contenido libres de los efectos de la deseabilidad social. Los resultados señalan que las medidas de agresividad se ven fuertemente afectadas por la deseabilidad social pero que las relaciones entre las mismas con la impulsividad se deben al contenido de agresividad de los tests y no a un factor común de deseabilidad social (AU)

The impact of social desirability on psychometric measures of aggression.

Although many studies have focused on the effects of social desirability in personality measures, few have analysed its effects on such highly undesirable behaviour as aggressiveness. The present study analyzes the impact of social desirability on measures of direct and indirect aggression and on the relationships between both kinds of aggression with impulsivity, using a method that enables the content factors of the measures to be isolated from social desirability. Results showed that aggression measures are highly affected by social desirability and that the relationships between the two forms of aggression and impulsivity are due to the content measured by the tests and not to a common social desirability factor.

Evaluación de las diferencias individuales en falseamiento / Assessment of individual differences in faking

En los últimos años, las puntuaciones de cambio obtenidas entre condiciones neutrales y condiciones de presión se han convertido en una de las principales alternativas para medir el falseamiento. Una importante cuestión aún pendiente relacionada con dichas puntuaciones es la relevancia de las diferencias individuales cuando las condiciones de presión son similares. El presente estudio propone un procedimiento, relativamente simple, y basado en la teoría clásica del test, para evaluar de forma rigurosa esta cuestión. El procedimiento, del que se derivan tres índices, se basa en un diseño pre-test post-test con grupo control y modela la magnitud de cambio como un parámetro individual. La propuesta se aplica en un estudio empírico en personalidad que proporciona resultados iniciales de interés (AU)

In recent years, change scores obtained under neutral conditions and under faking-inducing conditions have become one of the main alternatives for operationalizing faking. A pending issue regarding these measures is the relevance of individual differences under similar conditions of pressure. This study proposes a simple approach based on the classic test theory that allows the issue to be rigorously assessed. The approach, from which three indices are derived, is based on a pre-test post-test design with a control group, and models the amount of change as an individual parameter. The proposal is applied to an empirical study in personality, and some interesting initial results are obtained (AU)

[Assessment of individual differences in faking]. / Evaluación de las diferencias individuales en falseamiento.

In recent years, change scores obtained under neutral conditions and under faking-inducing conditions have become one of the main alternatives for operationalizing faking. A pending issue regarding these measures is the relevance of individual differences under similar conditions of pressure. This study proposes a simple approach based on the classic test theory that allows the issue to be rigorously assessed. The approach, from which three indices are derived, is based on a pre-test post-test design with a control group, and models the amount of change as an individual parameter. The proposal is applied to an empirical study in personality, and some interesting initial results are obtained.

Evaluación TRI de la escala I-E de Rotter: un nuevo enfoque y algunas consideraciones / An IRT assessment of Rotter’s I-E scale: a new approach and some considerations

Se evalúa el funcionamiento y propiedades de medida de la versión española de la escala de Locus de Control de Rotter; dicha evaluación utiliza un nuevo enfoque basado en la teoría de respuesta al ítem. Nuestro procedimiento permite: (a) detectar el impacto de los sesgos de respuesta (aquiescencia y deseabilidad social); (b) evaluar de forma clara la dimensionalidad y estructura del test; (c) evaluar las posibles causas del deficiente funcionamiento de algunos ítems; y (d) incrementar la precisión de la medida. Globalmente, los resultados no contradicen lo obtenido en estudios previos, pero permiten matizar algunas de las críticas que ha recibido la escala. Se proponen algunas recomendaciones para su uso (AU)

This study assessed the functioning and measurement properties of Rotter’s Locus of Control Scale in its Spanish version by using a new approach based on Item Response Theory. Our procedure allows us: (a) to detect the impact of response biases (acquiescence and social desirability); (b) to evaluate clearly the test dimensionality and structure; (c) to evaluate and understand the causes of some items’ inefficiency; and (d) to increase measurement precision. Globally, the results do not contradict those obtained in previous research, but allow some of the criticisms the scale had received to be qualified. Some recommendations for use are proposed (AU)

[An IRT assessment of Rotter's I-E scale: a new approach and some considerations]. / Evaluación TRI de la escala I-E de Rotter: un nuevo enfoque y algunas consideraciones.

This study assessed the functioning and measurement properties of Rotter's Locus of Control Scale in its Spanish version by using a new approach based on Item Response Theory. Our procedure allows us: (a) to detect the impact of response biases (acquiescence and social desirability); (b) to evaluate clearly the test dimensionality and structure; (c) to evaluate and understand the causes of some items' inefficiency; and (d) to increase measurement precision. Globally, the results do not contradict those obtained in previous research, but allow some of the criticisms the scale had received to be qualified. Some recommendations for use are proposed.

Assessing indirect aggression in aggressors and targets: spanish adaptation of the indirect aggression scales / Evaluando la agresividad indirecta en agresores y víctimas: adaptación española de las Escalas de Agresividad Indirecta

In recent years, there has been increasing interest in indirect aggression as the most common aggressive behaviour in adulthood. Despite this interest, there are not a great many instruments for measuring this behaviour in adults. The aim of our study was to develop the Spanish adaptation of one of the few instruments that does exist: the Indirect Aggression Scale, in its aggressor and target versions. The analysis of these scales in a sample of 935 university students showed that the aggressor and target versions of the scales had good reliabilities, but that a one-factor structure seemed more feasible than the three-factor structure initially proposed. Taking this one-dimensionality, we developed short versions of the scales, which also showed good reliabilities. The aggressor version presented good convergent validity with direct aggression and impulsivity measures. Finally, none of the scales showed differences associated with sex (AU)

En los últimos años se ha producido un creciente interés en la agresividad indirecta debido a que es la forma de agresividad más frecuente en la edad adulta. A pesar de ello no existe un gran número de instrumentos de medida para adultos de la misma. El principal objetivo del presente estudio es desarrollar una adaptación en español de uno de los pocos instrumentos disponibles: las Escalas de Agresividad Indirecta en sus versiones de agresor y víctima. El análisis de dichas escalas en una muestra de 935 estudiantes universitarios mostró que ambas formas presentan una buena fiabilidad pero que en ambos casos la estructura factorial de las mismas es unidimensional, en lugar de la estructura de tres factores propuesta por los autores. Teniendo esto en cuenta se plantea la posibilidad de desarrollar una escala reducida de un menor número de ítems. La versión para agresores presentó una buena validez convergente con otras medidas de agresividad y de impulsividad. Finalmente, no se observaron diferencias asociadas al sexo en ninguna de las escalas (AU)

Assessing indirect aggression in aggressors and targets: Spanish adaptation of the Indirect Aggression Scales.

In recent years, there has been increasing interest in indirect aggression as the most common aggressive behaviour in adulthood. Despite this interest, there are not a great many instruments for measuring this behaviour in adults. The aim of our study was to develop the Spanish adaptation of one of the few instruments that does exist: the Indirect Aggression Scale, in its aggressor and target versions. The analysis of these scales in a sample of 935 university students showed that the aggressor and target versions of the scales had good reliabilities, but that a one-factor structure seemed more feasible than the three-factor structure initially proposed. Taking this one-dimensionality, we developed short versions of the scales, which also showed good reliabilities. The aggressor version presented good convergent validity with direct aggression and impulsivity measures. Finally, none of the scales showed differences associated with sex.

The impact of acquiescence on forced-choice responses: a model-based analysis

El objetivo general del presente estudio es evaluar la potencial utilidad que tiene el formato de elección forzosa para reducir el impacto de la respuesta aquiescente. Las contribuciones con vistas a conseguir este objetivo son de dos tipos: metodológicas y substantivas. Metodológicamente, se propone un procedimiento basado en un modelo de análisis que, a su vez, se desarrolla a partir de un mecanismo de respuesta. Dicho procedimiento permite una evaluación rigurosa del impacto de la aquiescencia. Substantivamente, el procedimiento propuesto se aplica a tres cuestionarios con formato de elección forzosa ampliamente utilizados en personalidad: La escala de Locus de Control de Rotter, la escala de búsqueda de sensaciones de Zuckerman y la escala de aumentaciónreducción de Vando. Los resultados sugieren que el impacto de la aquiescencia es mínimo excepto en el caso de la escala de Rotter. Estos resultados diferenciales podrían explicarse por ciertas características de la escala que se discuten en el artículo(AU)

The general aim of the present study is to assess the potential usefulness of the normative Forced Choice (FC) format for reducing the impact of acquiescent responding (AR). To this end it makes two types of contributions: methodological and substantive. Methodologically, it proposes a model-based procedure, derived from a basic response mechanism, for assessing the impact of AR. Substantively, it applies the procedure in three large datasets, which use well-known normative FC personality questionnaires: Rotter’s Locus of Control Scale, the Sensation Seeking Scale (form V), and Vando’s Reducer-Augmenter Scale. The results suggest that the impact of AR is minimal except for the Locus of Control Scale, so FC-based questionnaires are a good alternative for controlling acquiescence. The different results obtained with the LOC Scale might be explained for some particularities of this scale which are discussed in the paper(AU)

[Assessment of depressive symptoms in old visually-impaired people: a study of the Geriatric Depression Scale]. / Evaluación de la sintomatología depresiva en población mayor con pérdida visual: un estudio de la Escala de Depresión Geriátrica.

This study has two main purposes: (a) to assess the functioning of the Geriatric Depression Scale in its Spanish version in old visually-impaired people, and (b) to assess the relevance of measuring depressive symptoms within a comprehensive assessment of psychological adjustment related to vision impairment in that population. Respondents were 329 people affiliated to the ONCE who were a representative sample of the target population. The assessment of the first goal let us to establish two cut-off points, and led to new methodological developments that combine Item Response Theory and Classical Test Theory. The results indicated that the scale is particularly useful in the target population and that it is very relevant in the assessment of psychological adjustment related to visual impairment in old people.

El análisis factorial como técnica de investigación en psicología / No disponible

El presente texto explica los principales aspectos del análisis factorial como instrumento de investigación psicológica. Se revisan en primer lugar los aspectos básicos a nivel conceptual, de modo que su lectura sea adecuada tanto para el lector principiante como para aquellos que quieran profundizar más en sus conocimientos de la técnica. Después, se discuten con cierto detalle las diferencias entre el análisis exploratorio y confirmatorio, y los procedimientos para estimar el modelo y obtener la solución transformada. Estos puntos se discuten siguiendo cada una de las etapas recomendadas en una investigación: desde el diseño y recogida de datos hasta la interpretación de la solución final. Finalmente, se explica el funcionamiento del programa Factor, un programa más completo que la mayor parte de los programas comerciales, y que además es de distribución libre. Adicionalmente, se propone al lector la realización de un ejercicio resuelto, a modo de práctica para la aplicación de lo expuesto en el texto (AU)

The present text explains the main aspects of factor analysis as a tool in psychological research. First, the basic issues are revised at a conceptual level, so that the review is appropriate for beginners as well as for those who want an in deep knowledge of the technique. Next, the differences between exploratory and confirmatory analysis as well as the procedures for fitting the model and transforming the initial solution are discussed in detail. These issues are discussed according to the recommended steps in a factor-analytic research: from the design and data collection to the interpretation of the final solution. Finally, the functioning of the “Factor” program is explained. Factor is more complete than most commercial programs, and is also freely distributed. Additionally, a solved exercise is proposed as a practice to apply the material discussed in the text (AU)

Evaluación de la sintomatología depresiva en población mayor con pérdida visual: un estudio de la Escala de Depresión Geriátrica / Assessment of depressive symptoms in old visually-impaired people: a study of the Geriatric Depression Scale

El presente estudio tiene dos objetivos principales: (a) evaluar la adecuación y funcionamiento de la Escala de Depresión Geriátrica en su versión española en la población mayor con déficit visual, y (b) evaluar la relevancia que tiene la medida de la sintomatología depresiva dentro de un proceso global de evaluación del ajuste a la pérdida visual en esta población. Participaron 329 personas afiliadas a la ONCE que forman una muestra representativa de la población de interés. La evaluación del primer objetivo lleva al establecimiento de puntos de corte e implica nuevos desarrollos metodológicos que combinan la Teoría de Respuesta al Ítem y la teoría clásica de los tests. Los resultados indican que la escala resulta particularmente útil en la población de interés y que tiene un peso importante en la evaluación del ajuste a la pérdida visual en la población de mayores con déficit visual (AU)

This study has two main purposes: (a) to assess the functioning of the Geriatric Depression Scale in its Spanish version in old visually-impaired people, and (b) to assess the relevance of measuring depressive symptoms within a comprehensive assessment of psychological adjustment related to vision impairment in that population. Respondents were 329 people affiliated to the ONCE who were a representative sample of the target population. The assessment of the first goal let us to establish two cut-off points, and led to new methodological developments that combine Item Response Theory and Classical Test Theory. The results indicated that the scale is particularly useful in the target population and that it is very relevant in the assessment of psychological adjustment related to visual impairment in old people (AU)